Add first-class skipped node outcomes - #21
Conversation
There was a problem hiding this comment.
Bottom line
REQUEST CHANGES — skip compaction is not shared by provider matching and Python binding, so a skipped branch can hide a live Stream input. Three other public-boundary inconsistencies make accepted skip outcomes non-durable or misbound. Reviewed exact head a2e1cf2c4c61ba5cc59626a75f78617bd413fff2 against merge base c1248186551a3d56b57c62ae14f1eb46397f9466.
Findings
-
High — partial skipped fan-in selects the wrong Stream parent (
src/avalanche/dag.py:1657, provider selection near:2671). Skip removal happens only during worker-side Python binding, afterStreamhas selected a positional parent from the unfiltered list. Reproduced with(skipped() & live_append()) >> consume(df=Stream(...)): the workflow returnedNoneand bound Stream to('skipped',)instead of the liveAppendResult. Use one skip-compacted logical-slot plan for implicit Python and upstream-consuming providers, preserving worker-side Ray selection without driver payload fetches. -
High — accepted metadata can make a successful skip impossible to read (
src/avalanche/types.py:30,src/runtime/operator/convert.py:96,src/runtime/executor.py:77).ava.skip()accepts arbitrary mapping values, but operator serialization later requires JSON. Reproduced:Pathandbytesmetadata are accepted, thennode_state_to_protoraisesTypeError; integer keys change on round trip, empty reason becomes absent, and unbounded metadata rides the small Ray status channel. Validate and recursively clone a bounded, strict JSON-safe schema atskip()with a nonblank bounded reason. -
Medium — skipped explicit positional-only inputs shift later values (
src/avalanche/dag.py:1679). Removing the skipped element before call reconstruction compacts later explicit values into earlier slots. Independent audit reproducedconsume(skipped(), live())againstconsume(a="A", b="B", /)as("LIVE", "B"). Preserve logical parameter identity, use the skipped slot's default when available, and reject skipped required positional-only slots clearly. -
Medium — per-slot multi-return skips silently lose skip status (
src/avalanche/types.py:45). Reproduced: anum_returns=2node returning(ava.skip("left missing"), "RIGHT")emits success and returns(None, "RIGHT"); the reason is not durable. Issue #13 defines a node-level outcome. Reject nested or per-slotSkipOutcomevalues for v1, or explicitly design slot-level status before accepting them.
Execution-service behavior
Skipped tasks still finalize and produce receipts; receipt fan-in remains separate from payloads, and Ray observes small status refs rather than task payloads. The focused skip/operator/TUI/execution-services suite passed 28/28, including real Ray. Preserve these properties while fixing the four boundaries above.
Merge and evidence state
The live PR was CONFLICTING; merge-tree conflicts were in CHANGELOG.md and src/tui/widgets/log_panel.py. Resolve against current main while preserving virtualized rendering and authored-skip display, then rerun focused TUI/operator tests. Missing regressions: partial/all-skip Stream fan-in under Local and real Ray, positional-only skips, per-slot skips, and hostile or oversized metadata.
Summary
ava.skip(reason, metadata=None)as a successful non-value node outcomeVerification
uv run ruff check src/ test/— OKuv run pytest test/skipped_outcome_test.py test/operator_tests/test_operator.py::TestOperatorLifecycle::test_authored_skip_is_recorded_and_run_succeeds test/tui_test.py::TestLogTimestamps::test_log_panel_renders_authored_skip_reason_and_metadata -q— 8 passeduv run pytest test/dag_test.py test/workflow_execution_test.py test/workflow_data_passing_test.py test/rerun_test.py test/operator_tests/test_hooks.py test/operator_tests/test_operator.py test/operator_tests/test_grpc.py -m 'not ray and not tmux' -q— 160 passed, 28 deselecteduv run pytest test/workflow_execution_test.py test/workflow_data_passing_test.py test/operator_tests/test_hooks.py -m ray -q— 8 passed, 13 deselecteduv run pytest test/smoke_test.py test/example_smoke_test.py test/operator_example_discovery_test.py -v— 11 passedCloses #13